2016.09.13
Revision Control Smells
As software developers, we talk about
code smells,
design smells, and even
configuration smells.
While reviewing some code, it occurred to me that
revision control smells are also distinguishable and important.
Here are some obvious ones and my recommendations for avoiding them.
An orderly revision control repository is a sign of professionalism.
Continue reading "Revision Control Smells"Last modified: Thursday, September 15, 2016 1:52 pm
2015.05.22
Grady Booch on the Future in Software Engineering
I was privileged to hear Grady Booch deliver a keynote on the Future in Software Engineering.
Here are my notes of some important statements and interesting soundbytes.
Continue reading "Grady Booch on the Future in Software Engineering"Last modified: Monday, May 25, 2015 7:35 am
2014.09.25
First, Do No Harm
Let’s face it: not all software developers are superstar programmers (and, trust me, not all luminary developers program in a sane way.) This means that when we maintain existing code, we must be very careful to avoid breaking or degrading the system we work on. Why? Because a failure of a running system can affect operations, people, profits, property, and sometimes even lives. Here are the rules.
Continue reading "First, Do No Harm"Last modified: Thursday, September 25, 2014 10:32 am
2014.07.30
Service Orchestration with Rundeck
Increasingly, software is provided as a service. Managing and controlling the service’s provision is tricky, but tools for service orchestration, such as Rundeck, can make our lives easier. Take software deployment as an example. A well-run IT shop will have automated both the building of its software using tools like make, Ant, and Maven and the configuration of the hosts the software runs on with CFEngine, Chef, or Puppet (see the post “Don’t Install Software by Hand”). Furthermore, version control tools and continuous integration will manage the software and the configuration recipes, handling developer contributions, reviews, traceability, branches, logging, and sophisticated workflows. However, these tools still leave a gap between the software that has been built and is ready to deploy, and the server that has been configured with the appropriate components and libraries and is ready to run the software.
Continue reading "Service Orchestration with Rundeck"Last modified: Monday, August 10, 2015 7:04 pm
2014.04.24
Developing in the Cloud
Running a top-notch software development organization used to be a capital-intensive endeavor, requiring significant technical and organizational resources, all managed through layers of bureaucracy. Not anymore. First, many of the pricey systems and tools that we developers need to work effectively are usually available for free as open source software. More importantly, cheap, cloud-based offerings do away with the setup, maintenance, and user support costs and complexity associated with running these systems. Here are just a few of the services and providers that any developer group can easily tap into
(you can find many more listed here):
Continue reading "Developing in the Cloud"Last modified: Thursday, April 24, 2014 4:13 pm
2014.01.15
Bespoke Infrastructures
In the 1920s, the Ford Motor Company embarked on an ill-fated attempt to establish an industrial town in an Amazon rainforest as a way to secure a cultivated rubber supply for its cars’ wheels. At the time, it already owned ore mines, forests, and a steel foundry to produce the raw materials for its cars; today, it buys from external suppliers, even its cars’ electronic control units. How do these two phases of the automotive industry’s history relate to the way we currently develop and adopt infrastructure in our profession?
Continue reading "Bespoke Infrastructures"Last modified: Saturday, March 1, 2014 2:05 pm
2013.12.06
The Frictionless Development Environment Scorecard
The environment we work in as developers can make a tremendous difference on our productivity and well-being. I’ve often seen myself get trapped in an unproductive setup through a combination of inertia, sloth, and entropy. Sometimes I put-off investing in new, better tools, at other times I avoid the work required to automate a time-consuming process, and, also, as time goes by, changes in my environment blunt the edge of my setup. I thus occasionally enter into a state where my productivity suffers death by a thousand cuts. I’ve also seen the same situation when working with colleagues: cases where to achieve a simple task they waste considerable time and energy jumping through multiple hoops.
Continue reading "The Frictionless Development Environment Scorecard"Last modified: Friday, December 6, 2013 3:49 pm
2013.09.10
Differential Debugging
If estimating the time needed for implementing some software is difficult, coming up with a figure for the time required to debug it is nigh on impossible. Bugs can lurk in the most obscure corners of the system, or even in the crevices of third-party libraries and components. Ask some developers for a time estimate, and don’t be surprised if an experienced one snaps back, “I’ve found the bug when I’ve found the bug.” Thankfully, there are some tools that allow methodical debugging, thereby giving you a sense of progress and a visible target. A method I’ve come to appreciate over the past few months is differential debugging. Under it, you compare a known good system with the buggy one, working toward the problem source.
Continue reading "Differential Debugging"Last modified: Wednesday, September 11, 2013 0:12 am
2013.07.25
Portability: Goodies vs. the hair shirt
“I don’t know what the language of the year 2000 will look like, but I know it will be called Fortran”
— Tony Hoare
Writing code that can run on any platform used to be a golden standard, as attested by the tens of books with the word “portable” in their title. Every day however staying true to the faith of portable code is becoming more challenging as mighty ecosystems amass resources to tempt us into their platform-specific version of heaven. We can write non-portable code out of laziness or ignorance, because we can’t be bothered to verify or check that our code follows a standard. We can also decide to write non-portable code following a pragmatic cost-benefit analysis. Let’s follow this approach and examine portability as a tool, looking at what we gain through it, the price we pay for it, and how we can cope with the challenge of upholding it.
Continue reading "Portability: Goodies vs. the hair shirt"Last modified: Thursday, July 25, 2013 1:00 pm
2013.06.19
How to Create Your Own Git Server
Although I'm a happy (also paying) user of GitHub's offerings,
there are times when I prefer to host a private repository
on a server I control.
Setting up your own Git server can be useful
if you're isolated from the public internet,
if you're subject to inflexible regulations,
or if you simply want features different from those offered by GitHub
(and other similar providers).
Setting up a Git server on a Unix (Linux, Mac OS X, *BSD, Solaris, AIX)
machine isn't difficult,
but there are many details to observe.
Here is a complete guide.
Continue reading "How to Create Your Own Git Server"Last modified: Wednesday, June 19, 2013 2:20 pm
2013.05.08
Systems Software
Systems software is the low-level infrastructure that applications run on: the operating systems, language runtimes, libraries, databases, application servers, and many other components that churn our bits 24/7. It’s the mother of all code. In contrast to application software, which is constructed to meet specific use cases and business objectives, systems software should be able to serve correctly any reasonable workload. Consequently, it must be extremely reliable and efficient. When it works like that, it’s a mighty tool that lets applications concentrate on meeting their users’ needs. When it doesn’t, the failures are often spectacular. Let’s see how we go about creating such software.
Continue reading "Systems Software"Last modified: Sunday, August 10, 2014 3:32 pm
2013.03.14
Software Tools Research: SPLASH Panel Discussion
Written by Dennis Mancl and Steven Fraser
At the recent SPLASH (Systems, Programming, Languages and Applications: Software for Humanity) conference, one of us (Steven Fraser) organized an international group of experts to discuss challenges in software tools research.1 The panelists included Kendra Cooper (University of Texas, Dallas), Jim “Cope” Coplien (Gertrud & Cope), Junilu Lacar (Cisco Systems), Ruth Lennon (Letterkenny Institute of Technology), Diomidis Spinellis (Athens University of Economics and Business), and Giancarlo Succi (Free University of Bolzano-Bozen).
The discussion interwove three threads—tool use, development, and education—and the panelists took a critical look at how well tools serve the needs of software professionals, managers, and academics. Their passion for the topic was reflected through some heated exchanges, even during the opening statements.
Continue reading "Software Tools Research: SPLASH Panel Discussion"Last modified: Sunday, June 16, 2013 2:13 pm
2013.01.23
The Importance of Being Declarative
A declarative programming style focuses on what you want your program to do rather than how to perform the task. Through diverse programming techniques, libraries, and specialized languages, you end up with code that sidesteps nitty-gritty implementation details, dealing instead with a task’s big picture.
Continue reading "The Importance of Being Declarative"Last modified: Wednesday, January 23, 2013 6:27 pm
2012.12.19
APIs, Libraries, and Code
Let’s say you want to display a JPEG-compressed image, calculate Pearson’s correlation coefficient, parse an XML file, or create a key-value store. You can often choose between using the functionality of the application’s platform (Java EE or .NET), calling one of several available external libraries, or writing the code on your own. It isn’t an easy choice because you have many factors to consider. Specifically, you must take into account the task’s complexity, as well as the licensing, quality, and support of competing alternatives. See how you can narrow down your choice by eliminating alternatives at the earliest possible decision point.
Continue reading "APIs, Libraries, and Code"Last modified: Wednesday, December 19, 2012 12:44 am
2012.12.05
Programming Languages vs. Fat Fingers
A substitution of a comma with a period in project Mercury's working Fortran code compromised the accuracy of the results, rendering them unsuitable for longer orbital missions.
How probable are such events and how does a programming language's design affect their likelihood and severity?
In a paper I recently presented at the
4th Annual International Workshop on Evaluation and Usability of Programming Languages and Tools
I showed results obtained by randomly perturbing similar programs written in
diverse languages to see whether the compiler or run-time system
would detect those changes as errors,
or whether these would end-up generating incorrect output.
Continue reading "Programming Languages vs. Fat Fingers"Last modified: Wednesday, December 5, 2012 11:40 am
2012.10.11
Virtualize Me
The virtual machine (VM) is the most dazzling comeback in information technology. IBM implemented a VM platform architecture in the late 1960s in its CP/CMS operating system. The company’s goal was to provide the time-sharing capabilities that its batch-oriented System/360 lacked. Thus a simple control program (CP) created a VM environment where multiple instances of the single-user CMS operating system could run in parallel. Thirty years later, virtualization was rediscovered when companies like VMware found ways to virtualize the less accommodating Intel x86 processor architecture. The popularity of Intel’s platform and the huge amount of software running on it made virtualization an attractive proposition, spawning within a decade tens of proprietary and open source virtualization platforms.
Continue reading "Virtualize Me"Last modified: Wednesday, December 19, 2012 12:21 am
2012.09.04
Don't Install Software by Hand
An IT system’s setup and configuration is a serious affair. It increasingly affects us developers mainly due to the proliferation and complexity of internet-facing systems. Fortunately, we can control and conquer this complexity by adopting IT-system configuration management tools.
Continue reading "Don't Install Software by Hand"Last modified: Tuesday, September 4, 2012 6:42 pm
2012.08.07
Importing an RCS Project into Git
RCS stands for Revision Control System.
You may have never heard it, because it's more than a quarter-century old;
a paper
describing it was published in 1985.
Although its commands are still available in most Unix distributions
and it's one of the easiest systems to use in a single-user scenario,
it is clearly showing its age when compared to more modern systems.
Here is how to move an existing project managed with RCS to the
21st century and Git,
while preserving all its history.
Continue reading "Importing an RCS Project into Git"Last modified: Tuesday, August 7, 2012 7:19 pm
2012.05.17
Git
Even by our field’s dizzying rate of progress I wouldn’t expect to revisit the subject of version control just six years after I first wrote about it in this column (Version Control Systems. Software, 22(5):108–109, September/October 2005). Yet here we are. The new kid on the block is git, a distributed revision control system available on all mainstream development platforms through a Free Software license. Git, a brainchild of Linus Torvalds, began its life in 2005 as the revision management system used for coordinating the development of the Linux kernel. Over the years its functionality, portability, efficiency, and third-party adoption have evolved by leaps and bounds to make it its category’s leader. (Two other systems with similar characteristics are Mercurial and Bazaar.)
Continue reading "Git"Last modified: Monday, September 3, 2012 11:19 am
2012.03.08
Package Management Systems
DLL hell was a condition that often afflicted unfortunate users of old Microsoft Windows versions. Under it, the installation of one program would render others unusable due to incompatibilities between dynamically linked libraries. Suffering users would have to carefully juggle their conflicting DLLs to find a stable configuration. Similar problems distress any administrator manually installing software that depends on incompatible versions of other helper modules.
Continue reading "Package Management Systems"Last modified: Thursday, May 17, 2012 3:35 pm
2012.01.11
Refactoring on the Cheap
The refactorings that a good integrated development environment can perform are impressive. Yet, there are many reasons to master some cheap-and-cheerful alternative approaches. First, there will always be refactorings that your IDE won’t support. Also, although your IDE might offer excellent refactoring support for some programming languages, it could fall short on others. Modern projects increasingly mix and match implementation languages, and switching to a specialized IDE for each language is burdensome and inefficient. Finally, IDE-provided refactorings resemble an intellectual straightjacket. If you only know how to use the ready-made refactorings, you’ll miss out on opportunities for other code improvements.
Continue reading "Refactoring on the Cheap"Last modified: Wednesday, January 11, 2012 6:23 pm
2011.10.30
Lessons from Space
By Diomidis Spinellis and Henry Spencer
We can lick gravity, but sometimes the paperwork is overwhelming.
— Wernher von Braun
Twilight saw the landing of Atlantis at the Kennedy Space Center on 21 July 2011, marking the end of the 30-year Space Shuttle program and leaving the Soyuz series of spacecraft as the only remaining major option for sending humans into space. With a history of 1,700 flights over an almost half-century lifetime, the Soyuz rocket and spacecraft are arguably a tremendously successful spaceflight design. Given the parallels between the complexity of human spaceflight and large software systems, what can we developers learn from the Soyuz program?
Continue reading "Lessons from Space"Last modified: Sunday, October 30, 2011 3:05 pm
2011.09.11
Faking it
This column is about a tool we no longer have: the continuous rise of the CPU clock frequency. We were enjoying this trend for decades, but in the past few years, progress stalled. CPUs are no longer getting faster because their makers can’t handle the heat of faster-switching transistors. Furthermore, increasing the CPU’s sophistication to execute our instructions more cleverly has hit the law of diminishing returns. Consequently, CPU manufacturers now package the constantly increasing number of transistors they can fit onto a chip into multiple cores—processing elements—and then ask us developers to put the cores to good use.
Continue reading "Faking it"Last modified: Sunday, September 18, 2011 6:43 pm
2011.07.23
How I Dealt with Student Plagiarism
Panos Ipeirotis,
a colleague at the
NYU Stern School of Business,
received considerable media attention when,
in a blog post he subsequently removed,
he discussed how his aggressive use of plagiarism detection software
on student assignments poisoned the classroom atmosphere and
tanked his teaching evaluations.
As detailed in
a story posted on the Chronicle of Higher Education blog,
Mr. Ipeirotis proposes instead that professors should design assignments that
cannot be plagiarized.
Along these lines here are two methods I've used in the past.
Continue reading "How I Dealt with Student Plagiarism"Last modified: Saturday, July 23, 2011 6:35 pm
2011.07.03
Agility Drivers
When the facts change, I change my mind. What do you do, sir?
— John Maynard Keynes
A management practice is mature when even government bureaucracies decide to adopt it.
The March 2011 publication of
UK’s ICT strategy
marks this moment by advocating that “the application of agile ICT delivery methods [...] will improve government’s capability to deliver projects successfully and realise benefits faster.”.
This begs the question: were we misguided during the decades we were advocating stringent control of requirements and a tightly milestone-driven development process? Interestingly, this was not the case. We were right then, and we’re right now. Things have changed, and this is why we can nowadays smugly apply agile practices reaping impressive dividends. Numerous new factors are driving agility by increasing our productivity. Our growing ability to swiftly put together sophisticated software affords us the luxury to listen to our customers, to try out new things, to collaborate across formal boundaries, to make mistakes, to redesign as we move along—in short to be agile. Knowing these factors helps us realize when we can afford to be agile and when not. (Hint: agile development of a plane’s flight control software from the ground up is still not a good idea.)
Continue reading "Agility Drivers"Last modified: Sunday, July 3, 2011 9:00 pm
2011.05.21
Code Verification Scripts
Which of my classes contain instance variables?
Which classes call the method userGet
,
but don't call the method userRegister
?
These and similar questions often come up when you want to verify
that your code is free from some errors.
For example, instance variable can be a problem in servlet classes.
Or you may have found a bug related to the
userGet
and userRegister
methods,
and you want to look for other places where this occurs.
Your IDE is unlikely to answer such questions,
and this is where a few lines in the Unix shell can save
you hours of frustration.
Continue reading "Code Verification Scripts"Last modified: Saturday, May 21, 2011 9:40 pm
2011.05.01
Choosing and Using Open Source Components
The developers of the SQLite open source database engine estimate that it’s deployed in roughly half a billion systems around the world (users include Airbus, Google, and Skype). Think of the hundreds of thousands of open source components, just one click away from you. If you know how to choose and use them effectively , your project can benefit mightily.
Continue reading "Choosing and Using Open Source Components"Last modified: Sunday, May 1, 2011 10:05 pm
2011.02.27
elytS edoC
Sure, you can write English right to left. You can also write software code to look like a disc or even a train (see www.ioccc.org/1988/westley.c and 1986/marshall.c). However, you can’t then complain when you have to fight with your magazine’s editor or production staff about accepting your column’s title for publication, or if your colleagues refuse to touch your code with a 10-foot pole. Writing code in a readable and consistent style is difficult, uninteresting, tedious, underappreciated, and, extremely important.
Continue reading "elytS edoC"Last modified: Sunday, February 27, 2011 8:49 pm
2010.10.30
Farewell to Disks
A classic web-comic illustrates how idle Wikipedia browsing can lead us from the Tacoma Narrows Bridge to Fatal hilarity (and worse). The comic doesn’t show the path leading from A to B, and finding it is an interesting challenge—think how you would engineer a system that could answer such questions. I believe that this problem and a solution I’ll present demonstrate some programming tools and techniques that will become increasingly important in the years to come.
Continue reading "Farewell to Disks"Last modified: Saturday, October 30, 2010 8:37 pm
2010.08.28
UML, Everywhere
flowchart, n.:
The innumerate misleading the illiterate.
— Stan Kelly-Bootle, “The Devil’s DP Dictionary”
A mechanical engineer who sees the symbol ⊥ in a diagram will immediately realize that a feature is specified to be perpendicular to another. In contrast, a software engineer looking at a diagram’s line ending with the symbol ◊ will, at best, wonder whether it denotes aggregation (as in UML), or a “zero or one” cardinality (as in IDEF1X), or something else invented by a creative academic. Worse, many developers will simply scratch their head in bewilderment.
Continue reading "UML, Everywhere"Last modified: Saturday, August 28, 2010 11:48 am
2010.07.11
Code Documentation
Technical prose is almost immortal.
— Frederick P. Brooks, Jr.
Although programming is a form of literary expression, the relationship between code and its documentation is uneasy at best. As Figure 1 shows, among the thousands of projects that FreeBSD maintainers have considered important enough to port to the platform, the number of comments per 100 lines varies substantially. Clearly, as developers our views on how we should document our code are anything but consistent. Yet, there are universal principles, nifty tools, and useful practices that can benefit us all.
Continue reading "Code Documentation"Last modified: Sunday, July 11, 2010 1:32 pm
2010.04.07
Ron Heifetz on Crisis Leadership
Earlier today I had the privilege to attend a lecture on
crisis management by the Harvard Senior Lecturer
Ron Heifetz.
Here is a list of points that struck me
(in the form of slightly edited tweets),
and my view of their relevance to software development.
Continue reading "Ron Heifetz on Crisis Leadership"Last modified: Wednesday, April 7, 2010 11:31 pm
2010.03.04
Software Tracks
A generous car reviewer might praise a vehicle’s handling by writing that it turns as if it’s running on railroad tracks. Indeed, tracks offer guidance and support. When you run on tracks you can carry more weight, you can run faster, and you can’t get lost. That’s why engineers, from early childhood to old age, get hooked on trains. Can we get our software to run on tracks?
Continue reading "Software Tracks"Last modified: Thursday, March 4, 2010 1:48 pm
2009.10.21
Basic Etiquette of Technical Communication
Parents spend years trying to teach their children to be polite, and some of us had to learn at school how to properly address an archbishop. Yet, it seems that advice on courteousness and politeness in technical communication is in short supply; most of us learn these skills through what is euphemistically called “on the job training.” With enough bruises on my back to demonstrate the amount and variety of my experience in this area (though not my skill), here are some of the things I’ve learned.
Continue reading "Basic Etiquette of Technical Communication"Last modified: Sunday, December 27, 2009 6:43 pm
2009.09.02
Job Security
My colleague, who works for a major equipment vendor, was discussing how his employer was planning to lay off hundreds of developers over the coming months. “But I’m safe,” he said, “as I’m one of the two people in our group who really understand the code.” It seems that writing code that nobody else can comprehend can be a significant job security booster. Here’s some advice.
Continue reading "Job Security"Last modified: Wednesday, September 2, 2009 3:35 pm
2009.06.09
Software Architecture Challenges in the 21st Century
A wonderful workshop, titled
Software Architecture Challenges in the 21st Century, took place
at the University of Southern California on June 8th.
The workshop was co-sponsored by IEEE Software,
USC's Center for Systems and Software Engineering,
and UC Irvine's Institute for Software Research (ISR).
Here is my personal summary of the talks; the presentation slides
are available
online.
Continue reading "Software Architecture Challenges in the 21st Century"Last modified: Tuesday, June 9, 2009 5:19 pm
2009.04.15
Drawing Tools
1 Word = 1 Millipicture
— /usr/games/fortune
It’s no accident that in all engineering branches, our colleagues often communicate using drawings and diagrams. Given many artifacts’ scale and complexity, a drawing is often the best way to describe them. Uniquely, in software development we can easily derive pictures from code, and sometimes even code from pictures.
Continue reading "Drawing Tools"Last modified: Wednesday, April 15, 2009 10:18 am
2009.02.25
Start With the Most Difficult Part
There’s not a lot you can change in the process of constructing a building. You must lay the foundation before you erect the upper floors, and you can’t paint without having the walls in place. In software, we’re blessed with more freedom.
Continue reading "Start With the Most Difficult Part"Last modified: Wednesday, February 25, 2009 2:58 pm
2009.02.10
Visualizing Revision Logs with a UML Sequence Diagram
How can you visualize the developers' contributions in a collaborative
project?
One way involves drawing timelines adorned with marks indicating
each developer's contribution.
This is a simple UML sequence diagram,
a diagram that allows you to see the interactions of objects,
but in this case the objects are the actual developers and the
interactions are their contributions.
Continue reading "Visualizing Revision Logs with a UML Sequence Diagram"Last modified: Tuesday, February 10, 2009 12:34 am
2009.02.04
Beautiful Architecture
What are the ingredients of robust, elegant, flexible, and maintainable software architecture?
Over the past couple of years, my colleague
Georgios Gousios
and I worked
on answering this question through a collection of intriguing essays
from more than a dozen of today's leading software designers and architects.
Continue reading "Beautiful Architecture"Last modified: Wednesday, February 4, 2009 1:48 pm
2009.02.03
The World's Smallest Domain-Specific Language
Domain-specific languages, also known as little languages, allow us
to express knowledge in a form close to the problem at hand.
In contrast to general-purpose languages, like Java or C++,
they are specialized for a narrow domain.
Earlier today I wanted to initialize a rectangular array of Boolean
values to represent the stick figure of a human.
For that I devised a tiny domain-specific language (DSL) consisting of
two symbols (representing an on and an off pixel) and wrote its
commensurably simple interpreter.
Continue reading "The World's Smallest Domain-Specific Language"Last modified: Tuesday, February 3, 2009 1:04 pm
2008.11.07
The Value of Computing Paradigm Diversity
Today I wrote a combinatorial optimization algorithm to match members of
pair programming
teams according to the psychological traits of each pair's members.
The program appeared to rearrange the initial random allocation of pairs
in a way that might match my specifications.
However, as I'll use this allocation for an experiment that I'll be able
to perform only once, I realized that I wanted to carefully verify the results.
How does one verify the operation of such a program?
Continue reading "The Value of Computing Paradigm Diversity"Last modified: Friday, November 7, 2008 6:03 pm
2008.10.18
A Look at Zero-Defect Code
The US
National Security Agency
has released a case study showing how to
develop zero-defect code in a cost-effective manner.
The researchers of the project conclude that,
if adopted widely, the practices advocated in the case study
could help make commercial software programs more reliable and less vulnerable.
I examined a small part of the case study's code, and was not impressed.
Continue reading "A Look at Zero-Defect Code"Last modified: Saturday, October 18, 2008 1:39 pm
2008.08.25
Saving the Editor's History
I recently spent a few days writing some tricky bit-twiddling code to
implement a radix tree.
I found myself making many programming mistakes, and I thought it would be
interesting to study them, examine their contributing factors, and
think how each of them could be prevented.
Continue reading "Saving the Editor's History"Last modified: Monday, August 25, 2008 4:32 pm
2008.06.26
The Way We Program
If the code and the comments disagree, then both are probably wrong.
— Norm Schryer
I can still remember the first time I laid my eyes on production-quality source code. This was in the early 1980s and the code was the BIOS listing of the original IBM PC. The 5940 lines of code spanned 80 neatly typeset pages in a three-ring slip-covered binder. Two things made a lasting impression on me. The first was the elation of being able to read, understand and learn from the code that made a real machine tick. This may have sparked my current practical and research interests in open source software. The second, was the way the code was commented. The BIOS was written in 8086 assembly language, and almost every line had a comment on its right hand side. By poring over the code and its comments I learned 8086 assembly, programming style, and understood the PC’s hardware architecture.
Continue reading "The Way We Program"Last modified: Thursday, June 26, 2008 12:40 am
2008.05.16
Open and Closed Source Kernels Go Head to Head
Earlier today I presented at the
30th International Conference on Software Engineering a
research paper comparing the
code quality of Linux, Windows (its
research kernel distribution),
OpenSolaris, and
FreeBSD.
For the comparison I parsed multiple configurations of these systems (more than ten million lines), and stored the results in four databases, where I could run SQL queries on them. This amounted to 8GB of data, 160 million records.
(I’ve made the databases and the SQL queries available
online.)
The areas I examined were file organization, code structure, code style, preprocessing, and data organization.
To my surprise there was no clear winner or looser, but there were interesting differences in specific areas.
Continue reading "Open and Closed Source Kernels Go Head to Head"Last modified: Friday, May 16, 2008 1:44 am
2008.05.02
Software Builders
The tools and processes we use to transform our system’s source code into an application we can deploy or ship were always important, but nowadays they can mean the difference between success and failure. The reasons are simple: larger code bodies, teams that are bigger, more fluid, and wider distributed, richer interactions with other code, and sophisticated tool chains. All these mean that a slapdash software build process will be an endless drain on productivity and an embarrassing source of bugs, while a high-quality one will give us developers more time and traction to build better software.
Continue reading "Software Builders"Last modified: Tuesday, May 6, 2008 12:04 am
2008.04.20
Assigning Responsibility
Over the past few days I worked over a large code body correcting various
accumulated errors and style digressions.
When I finished I wanted to see who wrote the original lines.
(It turned out I was not entirely innocent.)
Continue reading "Assigning Responsibility"Last modified: Sunday, April 20, 2008 9:34 pm
2007.12.28
Losing the War
The following gem comes from the user manual of the spanky-new Sony Ericsson
K770i (3G) mobile phone.
It demonstrates amply the state of the art in software engineering as
of 2007.
Continue reading "Losing the War"Last modified: Tuesday, January 1, 2008 4:51 pm
2007.11.29
An Update on UMLGraph
Today I released version 5.0 of UMLGraph.
This release contains a couple of important changes: six new shapes
(components, nodes, collaborations, use cases, notes, and active classes),
a facility for adding notes, and the hosting and packaging under its
own domain name umlgraph.org
.
Continue reading "An Update on UMLGraph"Last modified: Thursday, November 29, 2007 6:40 pm
2007.09.14
Cooperative Development at the Speed of Light
The agility and responsiveness of major open-source endeavors never
ceases to amaze me.
Continue reading "Cooperative Development at the Speed of Light"Last modified: Friday, September 14, 2007 9:42 pm
2007.09.02
Abstraction and Variation
“Master, a friend told me today that I should never use the editor’s copy-paste functions when programming,” said the young apprentice. “I thought the whole point of programming tools was to make our lives easier,” he continued.
The Master stroked his long grey beard and pressed the busy button on his phone. This was going to be one of those long, important discussions.
Continue reading "Abstraction and Variation"Last modified: Sunday, September 2, 2007 12:02 am
2007.06.28
The Tools we Use
It is impossible to sharpen a pencil with a blunt ax. It is equally vain to try to do it with ten blunt axes instead.
— Edsger W. Dijkstra
What’s the state of the art in the tools we use to build software? To answer this question I let over a period of a month a powerful server build from source code about seven thousand open-source packages. The packages I built form a subset of the FreeBSD ports collection, comprising a wide spectrum of application domains: from desktop utilities and biology applications to databases and development tools. The collection is representative of modern software, because, unlike say a random sample of sourceforge.net projects, these are programs that developers have found useful enough to spend effort to port to FreeBSD. The build process involves fetching each application’s source code bundle from the internet, patching it for FreeBSD, and compiling the source code into executable programs or libraries. Over the monthly period I also setup the operating system to write an accounting record for each one of the commands it executed. I then tallied up the CPU times of the 144 million records corresponding to the work in order to get a picture of how our software builds exploit the power of modern GHz processors.
Continue reading "The Tools we Use"Last modified: Sunday, September 2, 2007 12:06 am
2007.05.02
Of Wine and Software
Today, following the yearly IEEE Software
board meeting,
we visited the Casa Valduga winery near Bento Gonçalves at Rio Grande do Sul.
The guide told us that the winery nowadays made all its
wine in stainless steel vats, which have largely replaced the traditional
oak barrels.
Continue reading "Of Wine and Software"Last modified: Wednesday, May 2, 2007 3:02 pm
2007.04.30
Silver Bullets and Other Mysteries
It seemed like a good idea at the time.
—Ken Thomson, on naming the Unix system call to create a file "creat"
When conference participants interrupt a speaker with applause, you know the speaker has struck a chord. This happened when Alan Davis, past editor in chief of IEEE Software, gave a talk on improving the requirements engineering process at the NASSCOM (Indian National Association of Software and Services Companies) Quality Summit in Bangalore in September 2006. He was explaining why a marketing team will often agree with developers on additional features and a compressed delivery schedule that both sides know to be unrealistic. The truth is that this places the two parties in a Machiavellian win-win situation. When the product's delivery is inevitably delayed, the developers will claim that they said from the beginning that they couldn't meet the schedule but that marketing insisted on it. The marketing people also end up with a convenient scapegoat. If the product launch is a flop, they can say they missed a critical marketing time window owing to the product's delay. Where else are we playing such games?
Continue reading "Silver Bullets and Other Mysteries"Last modified: Monday, April 30, 2007 0:53 am
2007.04.21
Modularity and Troubleshooting
A residual current device trips leaving the house in the dark.
How do I fix the problem?
Continue reading "Modularity and Troubleshooting"Last modified: Saturday, April 21, 2007 7:06 pm
2007.03.15
Make vs Ant: Observability
I've long felt uncomfortable with ant
as a build management tool.
I thought that my uneasiness stemmed from the verbose XML used for
describing tasks, and the lack of default dependency resolution.
Today, email from a UMLGraph user
struggling with a complex ant task
made me realize another problem:
lack of observability.
Continue reading "Make vs Ant: Observability"Last modified: Thursday, March 15, 2007 5:04 pm
2007.03.09
Software Rejuvenation is Counterproductive
In the February issue of the Computer magazine
Grottke and Trivedi propose four strategies for
fighting bugs that are difficult to detect and reproduce.
Retrying an
operation and replicating software are indeed time-honored and practical
solutions. When coupled with appropriate logging, they may allow an
application to continue functioning, while also alerting its maintainers
that something is amiss. On the other hand, the proposal to restart
applications at regular intervals (rejuvenation as the authors call
it), doesn't allow us to find latent bugs, sweeping them instead under
the carpet. This lowers the bar on the quality we expect from software,
and will doubtless result in a higher density of bugs and increasingly
complicated failure modes.
Continue reading "Software Rejuvenation is Counterproductive"Last modified: Friday, March 9, 2007 3:37 pm
2007.02.27
A Peek at Beautiful Code
An exciting new book is about to hit the shelves,
and I consider myself very lucky to be among its contributors.
Beautiful Code,
subtitled "leading programmers explain how they think",
contains 33 chapters where contributors describe some code
they consider noteworthy.
Although I don't consider myself worthy of the book's subtitle,
I love coding, and
I'm extremely happy that code is taking the leading role among such an
illustrious cast.
Here is the complete table of the book's contents.
Continue reading "A Peek at Beautiful Code"Last modified: Tuesday, February 27, 2007 8:45 pm
2007.01.19
Open source as a paradigm for evolving complex systems
Scientists in the 1980s hotly debated the feasibility of US's proposed Strategic Defence Initiative, commonly known as Star Wars. One argument concerned the amount of software needed to control the missile detectors and weapons.
Continue reading "Open source as a paradigm for evolving complex systems"Last modified: Friday, January 19, 2007 9:33 am
2006.12.15
Cracking Software Reuse
[Newton] said, "If I have seen further than others, it is because I've stood on the shoulders of giants." These days we stand on each other's feet!
— Richard Hamming
Sometimes we encounter ideas that inspire us for life. For me, this was a Unix command pipeline I came across in the '80s:
Continue reading "Cracking Software Reuse"Last modified: Friday, December 15, 2006 12:31 am
2006.10.20
SQO-OSS Launched
Yesterday my research group and our partners officially launched the
SQO-OSS
research project:
a Software Quality Observatory for Open Source Software.
Continue reading "SQO-OSS Launched"Last modified: Friday, October 20, 2006 7:39 pm
2006.09.27
Choosing a Collection: A Discussion with Kent Beck
Recently I reviewed the mansucript of Kent Beck's upcoming
book Implementation Patterns.
I will certainly put it in the list of books any professional programmer
should read.
When discussing collections (containers in C++ STL parlance),
Kent mentions that
his overall strategy for performance coding with collections is to use the
simplest possible implementation at first and pick a more specialized collection
class when it becomes necessary.
My view is that
we should choose the most efficient implementation from the start.
With prepackaged collections this doesn't have any cost associated with
it, and it avoids nasty surprises when a dataset increases beyond the
size the programmer envisaged.
I added a comment to that effect in my review, and later I sent him
an email with a supporting citation, which
kindled an interesting exchange.
I reproduce our email exchange here, with his permission.
Continue reading "Choosing a Collection: A Discussion with Kent Beck"Last modified: Wednesday, September 27, 2006 3:36 pm
2006.09.25
The Verbosity of Object-Oriented Code
As I refactored a piece of code from an imperative to an
object-oriented style I increased its clarity and reusability,
but I also trippled its size.
This worries me.
Continue reading "The Verbosity of Object-Oriented Code"Last modified: Monday, September 25, 2006 0:32 am
2006.09.21
UML Class Diagrams from C++ Code
I needed a UML class diagram of the classes I use in the implementation of
CScout refactoring browser.
I drew the last such diagram on paper about four years ago, so it was
definitely out of date.
I always say that whenever possible documentation should be automatically
generated from the code, so I decided to automate the task.
Continue reading "UML Class Diagrams from C++ Code"Last modified: Thursday, September 21, 2006 11:49 am
2006.09.17
NASSCOM Quality Summit 2006
Last week I attended NASSCOM's 2006 Quality Summit in Bangalore, India.
There I gave a tutorial on tooling with open source software, and
delivered a talk on Global Software Development in the FreeBSD Project.
It was an edifying trip.
Continue reading "NASSCOM Quality Summit 2006"Last modified: Sunday, September 17, 2006 10:35 pm
2006.08.08
SeaMonkey vs Internet Explorer Revisited
In an older blog entry
I compared the complexity of Mozilla with that of Internet Explorer
by looking at the components each linked to.
Recently, comments to a Slashdot posting I made, noted
that I was comparing the two products on different platforms and
source with binary dependencies.
I therefore set out to compare the binary dependencies of the two
under Windows.
Continue reading "SeaMonkey vs Internet Explorer Revisited"Last modified: Tuesday, August 8, 2006 6:17 pm
2006.07.01
Choosing a Programming Language
A language that doesn't have everything is actually easier to program in than some that do.
— Dennis M. Ritchie
Computer languages fascinate me. Like a living person, each one has its own history, personality, interests, and quirks. Once you've learned one, you can use it again after years of neglect, and it's like reconnecting with an old friend: you can continue discussions from the point they were broken off years before. For a task I recently faced I adopted a language I hadn't used for 15 years, and felt enlightened.
Continue reading "Choosing a Programming Language"Last modified: Friday, December 15, 2006 12:32 am
2006.05.07
Surprising Findings on Software Reuse
Kevin DeSouza and his colleagues in a recent
article in the
Communications of the ACM published some surprising
findings regarding software reuse:
reuse happens more by novices rather than by experts,
within projects rather than across them, and in
transient teams rather than permanent ones.
The statement regarding the higher propensity of rookies to reuse
compared to older professionals rang particularly true to my ears.
Continue reading "Surprising Findings on Software Reuse"Last modified: Sunday, May 7, 2006 10:01 pm
2006.04.12
Code Quality: The Open Source Perspective
My new book
Code Quality: The Open Source Perspective
got published,
three years after I started writing it.
The book owes more to open source software than any of the books
dealing with Linux, PHP, Apache, Perl or any other book covering
a specific technology.
Continue reading "Code Quality: The Open Source Perspective"Last modified: Wednesday, April 12, 2006 12:05 am
2006.01.01
Project Asset Portability
It's said that real computer scientists don't program in assembler; they don't write in anything less portable than a number two pencil. Joking aside, at the end of the 1970s, the number of nonstandard languages and APIs left most programs tied to a very specific and narrow combination of software and hardware. Entire organizations were locked in for life to a specific vendor, unable to freely choose the hardware and software where their code and data would reside. Portability and vendor independence appeared to be a faraway, elusive goal.
Continue reading "Project Asset Portability"Last modified: Tuesday, December 12, 2006 9:20 pm
2005.09.02
Version Control Talk Demystified
One indication of the importance an
endeavor has in our lives is the vocabulary associated with it. If developers employ a tool or a method,
inevitably they will come up with words to describe their corresponding work in
an accurate and concise way. I recently
heard a colleague describe version control systems (also formally known as
configuration management tools) as boring. I hope that this dictionary
will dispel this myth by documenting a rich technical and social
vocabulary. If you don’t work with a
VCS I believe this list will give you plenty of reasons to look at what these
systems can do for you and your projects. On the other hand, if you already use a VCS I hope you will find ideas
on how to use it more productively and how to improve your configuration management
process. And, no matter to which group
you belong to, I am sure you’ll find here some new words worth knowing.
Continue reading "Version Control Talk Demystified"Last modified: Tuesday, December 12, 2006 9:20 pm
2005.09.01
Version Control Systems
A source code control system [is] a giant UNDO key—a project wide time machine.
— A. Hunt and D. Thomas
Sane programmers don't write production code without the help of an editor and an interpreter or a compiler, yet I've seen many software projects limping along without using a version control system. We can explain this contrast if we think in terms of the increased startup costs and the delayed gratification associated with the adoption of a VCS. We humans typically discount the future, and therefore implementing version control in a project appears to be a fight against the human nature. It is true that you can't beat the productivity boost that compilers and editors have provided us, but four decades after punched card programming in assembly language has gone out of fashion we must look elsewhere to reap our next gains in efficiency. And if you or your project is not using a VCS, adopting one may well be the single most important improvement you can undertake.
Continue reading "Version Control Systems"Last modified: Tuesday, December 12, 2006 9:20 pm
2005.07.01
Tool Writing: A Forgotten Art?
Merely adding features does not make it easier for users to do things—it just makes the manual thicker. The right solution in the right place is always more effective than haphazard hacking.
— Brian W. Kernighan and Rob Pike
In 1994 Chidamber and Kemerer defined a set of six simple metrics for object-oriented programs. Although the number of object-oriented metrics swelled to above 300 in the years that followed, I had a case where I preferred to use the original classic metric set for clarity, consistency, and simplicity. Surprisingly, none of the six open-source tools I found and tried to use fitted the bill. Most tools calculated only a subset of the six metrics, some required tweaking to make them compile, others had very specific dependencies on other projects (for example Eclipse), while others were horrendously inefficient. Although none of the tools I surveyed managed to calculate correctly the six classic Chidamber and Kemerer metrics in a straightforward way, most of them included numerous bells and whistles, such as graphical interfaces, XML output, and bindings to tools like ant and Eclipse.
Continue reading "Tool Writing: A Forgotten Art?"Last modified: Tuesday, December 12, 2006 9:20 pm
2005.03.26
UMLGraph Update
I have updated the UMLGraph program to
run under Java 1.5.
Continue reading "UMLGraph Update"Last modified: Saturday, March 26, 2005 6:29 pm
2005.02.11
An Open Source Java Metrics Toolset Is Hard to Find
I spent many hours looking for a stand-alone open source metrics toolset
for Java programs.
Unfortunately, I was not able to find a single tool that would
fit my (relatively modest) requiremets.
Continue reading "An Open Source Java Metrics Toolset Is Hard to Find"Last modified: Thursday, March 31, 2005 11:49 pm
2005.02.04
Maintainability of the FreeBSD System
Last November Ioannis Samoladas and his colleagues published an article
in the Communications of the ACM [1] that compared the maintainability
of open-source versus-closed source projects.
I applied the maintainability index [2] they used on the FreeBSD source
repository following the code's maintainability over time, and comparing
the maintainability of different modules.
Here are the results.
Continue reading "Maintainability of the FreeBSD System"Last modified: Friday, February 4, 2005 6:50 pm
2005.01.01
The Tools at Hand
The tools we use have a profound (and devious!) influence on our thinking habits, and, therefore, on our thinking abilities.
— Edsger W. Dijkstra
With a shovel excavator a single operator can effortlessly move 720 tons of earth with a single movement; a VLSI fabrication plant allows a designer to create elaborate sub-micron structures. Without tools the thousands employed in a car factory are nothing, with tools they can assemble a car in 18 effort hours. Sometimes, tools can even subsume the importance of their operators. The violinist Ivry Gitlis, considered one of the most talented musicians of his generation, said of his Stradivarius: "I have a violin that was born in 1713. I don't consider it my violin. Rather, I am its violinist; I am passing though its life." Tools are clearly an important and defining element of any profession and activity: tools help us move boulders and atoms, tools help us reach the Moon and our soul.
Continue reading "The Tools at Hand"Last modified: Tuesday, December 12, 2006 9:19 pm
2003.10.03
Software Complexity: Open Source vs Microsoft
In a readable and interesting paper titled
CyberInsecurity: the cost of a monopoly
seven notable security experts argue that the Microsoft's near monopoly
in the desktop operating system and office productivity markets is creating
a dangerous monoculture that exacerbates the effect of security vulnerabilities.
Continue reading "Software Complexity: Open Source vs Microsoft"Last modified: Friday, October 3, 2003 7:33 pm
2003.05.20
Code Reading: The Open Source Perspective
In July 2000, while working on a paper on the use of slicing for
choosing parts of an application to develop in a scripting language
(don't ask), I found myself searching open-source programs for
motivating examples, and experimenting with a tool for annotating the
corresponding source code. At some point, a loud click sound in my mind
brought to my attention the fact that although most books and courses
teach us how to program, we actually spend most of our time reading code
others have written. I reasoned that by applying my annotation tool on
open source software I could write a book to present the ideas,
techniques, and tools that go behind code reading.
Continue reading "Code Reading: The Open Source Perspective"Last modified: Friday, October 3, 2003 6:12 pm